1. Field of the Invention
The present invention relates to the field of graphical user interfaces for computer systems and, more particularly, to a method and apparatus for providing an intuitive methodology for an operator to control remote devices with a computer controlled object-oriented user interface utilizing animated graphic images.
2. Art Background
Most popular graphical user interfaces that are available today, such as the Xerox Star(copyright), Apple Macintosh(copyright), and Microsoft Windows graphical user interfaces, are based upon the xe2x80x9cdesktop metaphor.xe2x80x9d In a desktop metaphor system, the display screen is treated as a virtual desktop. Graphical symbols placed on the virtual desktop (i.e. the display screen) are used to represent common objects found in an office environment such as files, file folders, and printers. These desktop metaphor systems have several limitations which hamper the effectiveness of the man-machine interface between the human operator and the computer system.
One problem with the existing desktop metaphor graphical user interfaces is the use of a hierarchical file system. Most desktop metaphor graphical user interfaces are based upon a computer having a file system in which the computer files are organized in a hierarchical structure. The hierarchical file system structure has a root directory and several sub-directories within the root directory. Each sub-directory may have sub-sub-directories, and so on. The hierarchical file system structure is graphically illustrated on the screen as some type of root file cabinet with file folders inside. The Microsoft Windows and Apple Macintosh(copyright) operating systems both use hierarchical file systems illustrated as file folders.
The hierarchical structure of file folders is often very confusing to new users. For example, many users get lost while navigating through hierarchical file systems because the system of keeping files in file folders, which are in other file folders nested to an arbitrary depth is inherently confusing. Users become lost because each file folder looks like every other file folder, and the user is provided with little feedback as to his location within the file system hierarchy. Also, each file folder provides no information as to what the file folder contains other than a user-defined label. For example, except for a short text label, which-may have been defined by another user, the user generally is not able to determine whether an open folder is an accounting file folder containing spreadsheet data, or a data base file folder containing a customer data base. Therefore, navigating through the complex file system hierarchy is difficult since the current desktop metaphor graphical user interface offers little feedback as to a user""s location within the file hierarchy.
In current graphical user interface systems based upon the desktop metaphor, certain applications often do not fit well into a desktop metaphor environment conceptually. Specifically, non-office related applications may be inappropriate for a system based upon the desktop metaphor. For example, in a desktop metaphor, an application that controls a television would have to be placed into a xe2x80x9cpaperxe2x80x9d file folder on the desktop.
Another common problem with desktop metaphor user interfaces is that displaying a large number of applications on the desktop results in a cluttered desktop. Many users of desktop metaphor systems keep large numbers of applications and files, usually in icon form, on the display screen. Some attempt is made to use a graphical representation that identifies the type of application or file. However, when a user wishes to open a particular application or file, the user must search the display screen to find the desired icon just as if the user were trying to find a particular paper file on a messy desktop. Icons placed the desktop have no apparent hierarchy, and are all equal. This equality may confuse a user, and may create problems when the user wants to switch from one task to another.
Several systems have evolved to improve the organization of the desktop metaphor graphical user interface. One system is the concept of xe2x80x9cRoomsxe2x80x9d. See, Patrick Chan, xe2x80x9cLearning Considerations in User Interface Design: The Room Modelxe2x80x9d, Report CS-84-16, University of Waterloo Computer Science Department, Ontario, Canada, July, 1984; Brenda Laurel, xe2x80x9cComputers as Theatrexe2x80x9d, Addison-Wesley Publishing Co., 1991; Brenda Laurel, xe2x80x9cThe Art of Human-Computer Interface Designxe2x80x9d, Addison-Wesley Publishing Co., 1990; U.S. Pat. No. 5,072,412, Henderson, Jr. et al., xe2x80x9cUser Interface with Multiple Workspaces for Sharing Display System Objectsxe2x80x9d, and the references cited therein.
In systems based upon the Rooms concept, the user creates different rooms where the user performs a single task or a group of related tasks. Each room represents an individual virtual desktop with a group of associated tasks. Each room is given a label that identifies the room with the room""s functions. For example, a user may create a mail room, a typing room with a word processor, and an accounting room with a spreadsheet. Each room has only those icons and windows that are associated with the particular task of the room, as well as any windows shared by all rooms, such as a clock.
To switch between tasks in a system that uses the Rooms concept, a user moves between the different rooms that are associated with the different tasks. For example, to go from a spreadsheet to retrieve email, a user moves from the accounting room to the mail room. To move between the two rooms, a user selects an icon that represents a door connecting the current room to the other room. Users can create doors between any of the available rooms.
Although the Rooms concept helps solve the problem of the xe2x80x9ccluttered desktop,xe2x80x9d the Rooms concept still relies upon the common desktop metaphor. For example, the Rooms interface concept does not help simplify the confusing hierarchical file structure used to store information. The Rooms concept also does little to solve the form and function inconsistencies that are apparent when applications which do not fit into the desktop metaphor are added to the system.
In prior art graphical user interfaces, each time a new piece of hardware is introduced to the system a new piece of control software must also be added by the user. For example, if a user of an Apple(copyright) Macintosh(copyright) or a user of Microsoft(copyright) Windows desires to add a new printer to the system, the user must also install a new printer driver to control the added printer. Therefore, each time a user desires to add a new device to the system, the user must also follow a complicated procedure to add the appropriate software.
As will be described, the present invention provides an improved graphical user interface that overcomes several of the problems associated with desktop metaphor systems. A method and apparatus for obtaining the user interface of a remote device without requiring the user to load the related software is also disclosed. The remote device can be controlled using the remote device""s user interface as displayed on the present invention""s display screen. Although the user interface is described with reference to the control of remote devices, it will be appreciated from the discussion that follows that the user interface may be applied to a variety of applications in computer workstations, portable computers, and hand held control systems.
The present invention provides an intuitive graphical user interface and a method and apparatus for controlling remote devices. The graphical user interface of the present invention is displayed on a hand-held display device. The hand-held display device comprises at least one central processing unit (CPU) coupled to a graphic display system. In the present embodiment, the graphic display system is touch sensitive to permit the user to input information using a fingertip without the need for additional input devices such as a keyboard. A communications circuit is also coupled to the CPU transmitting information over some communication medium. The communications circuit is used to control and to communicate with remote devices.
The graphical user interface of the present invention creates a world of Spaces organized into a geographic map structure that is simple to navigate through since each Space provides familiar geographic surroundings. Each Space is represented by a background image which provides navigational landmarks that identify where the Space is located within the geographic map structure. For example, the background image of a living room Space contains items typically found in a living room such as a lamp, a chair, and a table. Therefore, navigation through this xe2x80x9cworldxe2x80x9d is made easier because the user can orient himself by looking around and examining the surroundings just as a person would do in the real world.
Within each Space, the user interface provides graphic objects that can be selected and manipulated by the user. The graphic objects are rendered as animated cartoon-like graphic objects that are easily identifiable as real world objects such as televisions and VCRs. There are at least three types of objects that exist in the Spaces of the user interface: data objects, buttons, and portals. Data objects are objects that are associated with a particular piece of data. Buttons are objects that are associated with a particular function or action; when a user selects a button, the user interface performs the function or action associated with the button. Portals are objects that are displayed in a first Space but are associated with a second Space; when a user selects a portal the user passes through the portal and moves to the second Space associated with that portal. The user environment is further enhanced by the use of sound, coordinated with every user action and supplementing the visual feedback provided.
To control real world remote devices, many of the graphic objects presented on the screen are associated with remote devices in the real world. To control a remote device, a user selects the graphic object pictured on the display screen associated with the particular remote device. After the user selects a graphic object associated with a remote device, the hand-held display device displays a user interface for controlling the remote device. The user can then interact with the remote device""s user interface. To control the remote device, the present invention invokes a device driver method for the remote device. The device driver method may be located within the remote device or within the local hand-held display device, depending on the remote device""s sophistication.